Document Thumbnail Visualization for Rapid Relevance Judgments: When do They Pay Off?

In a recent experiment, Kaugars (1989) developed a document visualization system that displayed the results of a keyword text search as a set of small graphical representations (thumbnails) of the top 20 retrieved documents laid out in one window and a single document viewer in a second window (See Figure 1). Document thumbnails presented color-coded highlighting indicating positions and identities of search terms in the document, and the single document viewer showed a fish-eye view of the document focused on multiple regions defined by the sentences holding search terms. He compared this system to another more traditional system that represented documents as lists of short titles and a simple scrolled window document viewer. The users judged 8 sets of documents in accordance with relevance judgments supplied with 8 selected TREC-6 topics. The documents and topics were all pre-selected for the users who had no part in selecting query terms or interacting with the retrieval software. Kaugars found that people were faster and better at making relevance judgments for a fixed set of retrieved documents when using the thumbnail/fish-eye system.